Managing Off-Chip Bandwidth: A Case for Bandwidth-Friendly Replacement Policy

With the expected increase in the number of on-chip cores the demand for off-chip bus, memory ports, and chip pins increases. This makes off-chip bandwidth a very scarce resource and can severely hurt performance. Off-chip bandwidth is mainly generated by the on-chip cache hierarchy (cache misses and cache writebacks), which depends on the replacement policy. There is a huge body of research on enhancing the cache replacement policy to reduce the number of misses of a cache. Bandwidth requirement has always been of secondary importance. In the multicore and manycore era this is no longer the case. Replacement policy must take into account both the cache performance and the traffic generated by the cache, which is the topic of this paper. In this paper we study several dynamic replacement policies with the aim to reduce the traffic generated from last level cache to main memory while not increasing the number of misses. We show that we can do better than a traditional LRU policy with very little hardware overhead.